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INTRODUCTION 

From 1973 to 1975, the Satellite Technology Demonstration (STD) implemented a compre- 
hensive series of steps — called the "formative process" to design, develop, and refine 
television programming for junior high school students. The programming was titled "Time 
Out!" 

This paper reviev/s the purpose for a formative process, as well as the resources needed 
to implement it, and describes briefly the step-by-step procedures involved in the process. 
The paper concludes by discussing the results of the STD's formative efforts and by pre- 
senting recommendations for similar future applications. 

PURPOSE 

The "Time Out!" student program, developed through the formative process, benefited 
junior high school students by increasing their knowledge of, and enhancing their attitudes 
toward, career-related concepts. To achieve these goals, the "Time Out!" program centered 
on three areas: decision-making, self-assessment, and job awareness. These areas were 
specified in 27 objectives. 

The formative process, implemented to insure that the "Time Out!" series achieved the 
prescribed objectives, required major investments in manpower, time, and money. But it 
was better than the alternative process of producing and distributing an untried product. 

If a product proved to be inappropriate or Ineffective, it would have to be revised. 
Revisions are expensive. Further, an untested product provides students with a haphazard — 
and often, ineffective — learning experience. The formative process, however, not only 
minimizes the risk that the program will be "off target," but also facilitates the inte- 
gration of resources, as well as identifies decision points for Project personnel. 
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RESOURCES 



Several resources are necessary to implement a formative process. Fo** the process to 
be viable, management and program personnel must be willing to support the revision of 
programs and program elements that are not achieving specified objectives. In short, a form- 
ative effort is based on corrmitment. Other necessary resources include: access to repre- 
sentative products; access to representative populations; and sufficient "lead time" for se- 
quential steps. 

STEP-BY-STEP PROCEDURES 

The formative process implemented by the STD involved 16 sequential steps. Figure 1 on 
the following page lists these steps. Note that the steps form four clusters: 

1. Identification and development of content (Steps 1.0 through 3.0). 

2. Development and refinement of scripts (Steps 4.0 through 8.0). 

3. Production and refinement of video products (Steps 9.0 through 13.0). 

4. Refinement of television programming during the operational period 
(Steps 14.0 through 16.0). 

Cluster 1: Content Development 

Figure 1 shows that the development of the "Time Out!" content began by identifying and 
clarifying user needs (Step 1.0). A comprehensive needs assessment was conducted to obtain 
input from State Department of Education personnels local school personnel, and junior and 
senior high school students. This input provided information on student interests, aware- 
ness, and needs in the area of career development, as well as on past exposure to televi- 
sion programming. 

The next step in the development of the "Time Out!" content involved creating a content 
structure and identifying appropriate formats and delivery vehicles (Step 2.0). Several age 
ranges and program formats were considered before the needs assessment. But fiscal limita- 
tions and preferences expressed by federal, state, and local personnel led to a decision in 
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spring, 1973, to aim programming at junior high school audiences. 

The needs assessment indicated that junior high school students were very interested in 
space-age technology; this helped in selecting an appropriate format. General content con- 
siderations included: 

1, Other needs assessment data; for example, educational surveys conducted by various 
state agencies, 

2, Available career education, development, and guidance models. 

3, Existing film resources to supplement the curriculum. 

These inputs helped the STD to design a curriculum featuring self-assessment, decision- 
making, and occupational awareness all of which were high priority topics for junior high 
school students. The central theme was time travel. 

Once the content for the overall series was specified, it needed to be organized into 
programs (Step 3,0); a spiral approach was used. The key to this spiral was occupational 
awarenesc, based on the Dictionary of Occupational Titles. With a spiral approach, all 
the occupations were presented briefly in the first program in the series; succeeding 
programs covered the occupations in greater detail. 

Content specifics were recorded in information memos. Each menio identified program ob- 
jectives, suggested general themes, and contained supplementary information for television 
script writers. 

Cluster 2: Script Development 

Professional television^script writers used the information memos to generate inci- 
dentSj appeals, and dialogue that would present- the desired content in an effective and 
acceptable manner (Step 4,0). The script produced at this step was comprehensive. It 
identfied all dialogue, settings, and action, as well as any existing material to be inte- 
grated into the program. 

The script now moved to Step 5,0. Here, it was reviewed by content and production per- 
sonnel. Questions answered included: 

1, Did the writers integrate the content in a manner free from distortion? 
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2. Did the writers leave out any salient content? 

3. Was the script, as presented, technically feasible? 

Because the STD used multidiscipline teams — composed of content and production per- 
sonnel -•- in the development of programs, ma^ r problems seldom were identified in Step 5.0. 
Those problems that did occur were solved quickly by revising the script (Step 6.0). At 
this point, the script was approved by content and production personnel and was ready for 
broader distribution and reaction. 

Step 7.0 included two reviews. An in-house review of the script by other Project per- 
sonnel was first. This gave personnel from management, research, and field services an op- 
portunity to offer suggestions; it helped to generate some meaningful feedback and, more 
importantly, to create product awareness and identification among the team. 

A review by persons outside the Project was second. Here, a panel of 72 people, nine 
per state, read the scripts and provided constructive criticism. Panel members consisted of: 
junior high school personnel (teachers, counselors, and administrators); State Department of 
Education personnel (career-development specialists and media specialists); and representa- 
tives from the Public Television Stations in the eight-state region. 

As scripts became available for field review, cross-section samples from the panel pop- 
ulation were drawn. This meant that no one panel member was responsible for reviewing all 
the scripts. As a result, panel members could devote more time -- and consequently, 
provide better input -- to the scripts they did review. 

The review by panel members yielded two types of data: objective and subjective. Not 
only did the members rate the scripts, using a scale provided by the STD, but they also in- 
serted their own comments and suggestions by pencil editing the scripts. This data was 
used to guide script revisions (Step 8.0). The revised scripts now were ready for produc- 
tion. 

Cluster 3: Video Production 

Because the STD's "Time Outi" program involved a compilation of segments and formats, 
it was impossible to produce each program as a separate entity. Instead, several episodes, 



using the same set and talent, were produced at once; these episodes then were integrated 
into the appropriate programs. The production of video segments and formats constituted 
Step 9.0. 

Once video segments were available, they were reviewed carefully by production and 
content personnel for technique, quality, and consistency. This review, Step 10.0, resulted 
in minor revisions; the revisions completed Step 11.0. 

After the initial in-house review and subsequent revision of scripts, video segments 
and some limited programs were presented to rural and urban junior high students and their 
teachers (Step 12.0). Because of temporal and fiscal constraints, only limited field 
review of video materials was possible (Step 13.0). Feedback from this review, however, 
resulted in some modification of segments, formats, and roles, as well as in a general en- 
dorsement of program style and content. It also helped to insure that the programs gener- 
ally were "on target." With this insurance, program production continued, thus completing 
the series for the third period. 



Cluster 4: Operational Status 

"Time Out!" ran one semester; thus, it was possible to distribute the series to an aud- 
ience the first semester, then to make revisions before showing it to a second audience the 
next semester. The first semester became a comprehensive field test, known as the "forma- 
tive semester" (Step 14.0). 

During the formative semester, an extensive data base was assembled to determine 
program acceptance and effectiveness and to guide subsequent revisions. Recommendations 
for revisions came from: daily and quarterly student and teacher acceptance ratings; content 
analysis of audio and written materials; in-house technical reviews of programs (by Project 
personnel); and out-of-house critical reviews of programs (by selected teachers). 

These recommendations were reviewed by content and production personnel, who then de- 
termined whether an element or program should be revised. Three priorities were established: 

1. Revise segments or characters which, in several programs, have a negative impact 
on the audience. 
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2. Revise programs v;hich, in several segments, have a low acceptance rating. 

3. Revise programs which are v/eak in one or more segments, but strong in others. 

Because general acceptance of the "Time Out!" series was high and because fiscal con- 
straints precluded revisions except where necessary, only modest revisions occurred between 
the two semesters (Step 15.0). These revisions dealt primarily with refining "voice-over" 
narrations of video film clips, smoothing transitions between segments, and modifying char- 
acterizations that received less than positive acceptance. The final version of the "Time 
Out!" series was distributed the second semester (Step 16.0). 

RESULTS 

The formative process was not defined clearly during the early stages of the Project; 
it should have been an integral part of the initial needs assessment and the development of 
a content structure, but it was not. 

In the early stages of the Project, formative efforts emerged pragmatically. In retro- 
spect, these efforts were clearly appropriate and in the proper sequence. Without a well- 
defined process or "master plan," however, it was difficult to determine — and even more 
difficult to communicate — where the "Time Out!" series was in its formative development. 

In early 1974, the 16-step flow chart described in this paper was defined. The cre- 
ation of a clear set of steps irrmediately enhanced both internal and external communica- 
tions. It enabled various functional components within the Project (prograrmiing, research, 
and utilization) to identify where and how they fit into the step-by-step process. Research 
and utilization, for example, were responsible for needs assessment (Step 1.0), field review 
of scripts (Step 12.0), and feedback from the formative semester (Step 14.0). With a clear 
delineation of the steps, everyone, regardless of component affiliation, was aware of the 
process being implemented. 

While awareness and coordination are both important results of the formative process, 
acceptance and effectiveness of programs determine the ultimate worth of the effort. The 
first-semester data indicates that the "Time OutI" series has received widespread student 
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and teacher acceptance. Equally positive has been student growth in decision-making, self- 
assessment, and occupational awareness, as measured by pre- and post-tests. 

RECOMMENDATIONS 

The STD implemented a formative process to design, develop, and refine television pro- 
gramming. Its experience suggests the following: 

1. Define the process, in-depth, before developing the product . This may SGem ob- 
vious, but it is difficult to do, especially when the organization is new, as 
indeed the STD was, and the personnel represent different disciplines, view- 
points, and priorities. 

2. Weigh the alternatives . Be sure that the process can adapt pragmatically to the 
resources and constraints of the situation. The STD's formative semester, for 
example, grew out of a need to compensate for studio delays, which precluded ade- 
quate early field testing of the programs, and to incorporate relevant research 
inputs, such as student and teacher acceptance data, into an operational setting. 

3. Test the process . Studio delays made it impossible to convert early scripts into 
video products and show them to script reviewers. As a reault, the STD relied 
heavily on the competence of its content personnel and the perceptions of its 
reviewers, the teachers and administrators. Did the junior high school students 
find the programs acceptable and useful? There was no way to answer that 
question. The STD was, however, fortunate; the script reviewers often were 
accurate' in assessing the students' interests, vocabulary levels, and career- 
related knowledge — based on their intuition, not on empirical research. 

4. Analyze the procedures . In retrospect, the STD should have committed itself to 
putting the early scripts into video production, even if this meant subcon- 
tracting for "out-of-house" production until "in-house" capability was established. 
Visual products would have provided the script review team with a context for 
their efforts; they also would have facilitated the earlier involvement of 
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actual users, the junior high school students. 
5. Verify the results . When the early programs finally were available, they were used 
to collect data on student and teacher acceptance of the product. But they did 
little to facilitate data on student learning, which could be tested only over 
time. Because "Time Out!" is an educational tool, this omission is serious. 
Content and production personnel relied heavily on acceptance data. Yet, the most 
accepted segments are not necessarily the most instructionally useful. For 
Instance, input from students and teachers during the operational period would 
have indicated that these groups felt the less animated and less acceptance- 
motivated segments had been the most useful. Again, the creation of one or two 
model programs, through out-of-house capability if necessary, would have been 
desirable. 

In sum, the key to developing a formative process is planning. If you state the 
problem clearly, the solution is self-evident. If you know the problem, you can develop 
al ternatives. 

The STD, for example, needed to know If its programs were "on target." But delays in 
video production and fiscal constraints, as mentioned before, made early field testing im- 
possible. As an alternative, the STD developed the formative semester. Although field 
testing, if possible, would have been the best measure of acceptance, the semester did 
prove useful. It provided an opportunity to test, in an operational setting, not only the 
video products, but also the supplementary print materials and evaluation tests and pro- 
cedures. Clearly, this step in the formative process should be given strong consideration 
by future users of a formative process. 

r^re specific information on the STD's formative efforts can ba obtained from the 
following technical reports: 

TR0211, Determining User Needs as a Basis for the Educational Progranm lnq of Large- 
Seal e Projects 

TR0213, The Contribution of an Internal Review Panel to the Developmen t of Educational 
Programming 
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other papers pertinent to this report include the following: 

TR0505, The Use of Courseware Teams for Achieving Content Objectives in Television 
Programming 

TR0506, Developing and Implementing a Content Structure for Educational Television 
Programming in the Area of Career Development 



This report was produced with funding from the National Institute 
of Education. The views expressed do not necessarily reflect those 
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Health, Education, and Welfare, 




